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1.  TECHNICAL  SUMMARY 


ABSTRACT 

This  project  used  analytical  and  experimental  techniques  derived  from  signal  detection 
theory  to:  (1)  quantify  the  decision-making  performance  of  individuals  and  teams;  particularly 
with  respect  to  how  information  received  from  sources  having  different  properties  is  chosen, 
weighed  and  combined,  (2)  model  the  decision  behavior  of  experienced  teams  of  human  operators 
operating  under  incentives  for  decision  speed  and  accuracy,  and  (3)  describe  how  the  accuracy 
and  speed  of  group  deliberation  depends  on  the  aggregation  rule  and  response  protocol  that 
constrains  the  sequential  order  of  information  exchange  among  team  members.  The  basic 
(individual  or  group)  decision  task  was  to  decide  on  the  presence  or  absence  of  signals  in  noise. 
Signals  were  presented  to  operators  on  individual  graphical  displays  and  the  team  then  had  to 
reach  a  decision  about  signal  occurrence.  The  project’s  experiments  show  how  individual  and 
team  performance  depends  on  team  member  signal-to-noise  ratio,  correlation  among  members’ 
inputs,  efficiency  of  member  updating  of  likelihood  estimates,  and  constraints  on  member 
interaction  and  communication.  Specifically,  the  project  experiments  demonstrated  that:  (a) 
members’  individual  estimates  are  combined  to  form  the  team’s  decision  with  high  efficiency, 
relative  to  the  optimal  Bayesian  rule,  (b)  team  performance  depends,  in  a  predictable  way,  on  the 
response  protocols  that  constrain  the  sequential  order  of  information  exchange  among  team 
members,  and  (c)  team  members  choose  (and  evaluate)  additional  sources  of  information  that 
vary  in  signal-to-noise  ratio,  bias,  cost,  time,  and  correlation  in  a  near  optimal  fashion.  These 
results  have  important  potential  applications  for  optimizing  team  decision  making,  particularly  for 
(a)  structuring  member  communication  so  as  to  minimize  the  effects  of  interruptions,  biases, 
correlations,  and  other  constraints,  (b)  enabling  the  optimal  aggregation  of  information  for  team 
decisions  at  maximum  accuracy  and  minimal  cost  and  time,  (c)  quantifying  and  training  expert 
team  behavior,  and  (d)  designing  the  most  effective  systems  for  spatially  separated,  networked 
teams  of  human  operators.  These  results  were  reported  in  two  Master’s  theses,  two  doctoral 
dissertations,  and  a  number  of  research  articles  and  presentations. 
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Basic  Theory  and  Experimental  Procedure 


This  research  (see,  Sorkin,  Luan  &  Itkowitz,  2004)  employed  an  approach  derived  from 
electrical  engineering  analyses  of  distributed  (or  decentralized)  signal  detection  (see  Pete  et  al., 
1993a,b;  Sorkin  &  Dai,  1994;  and  Viswanathan  &  Varshney,  1997)  to  model  individual  and  team 
decision  behavior.  This  approach  has  generally  not  received  attention  in  the  traditional  human 
judgment  and  decision  literature. 

In  the  classic  signal  detection  situation,  the  decision  maker  (DM)  makes  an  observation, 
x,  and  then  must  decide  whether  her  observation  was  caused  by  a  signal-plus-noise  or  a  noise- 
alone  event  (hereafter  referred  to  simply  as  ‘signal’  and  ‘noise’).  Her  ability  to  discriminate 
between  the  two  events  is  defined  by  the  two  distributions,  f(x\n),  the  likelihood  that  the 
observation  was  due  to  noise,  and  f(x\s),  the  likelihood  that  the  observation  originated  from 
signal.  An  ideal  (or  optimal)  decision  rule  is  the  likelihood  ratio,  l(x)=f(x\s)/f(x\n).  DM’s  ability  to 
discriminate  between  the  two  events  is  given  by  the  detection  parameter,  d\  the  normalized 
separation  between  the  means  of  the  two  (assumed)  normal  distributions,  as  shown  in  figure  1 . 

All  of  the  information  that  DM  has  about  the  existence  of  a  signal  on  any  trial  is  specified  by  the 
value  of  the  likelihood  ratio,  l(x).  These  distributions  are  sometimes  plotted  on  a  log-likelihood 
ratio  axis;  this  preserves  their  normal  shape,  generalizes  the  dimensions  of  the  decision  axis,  and 
simplifies  subsequent  updating  of  the  likelihood  estimate  by  additional  observations. 


By  defining  the  payoffs  (or  penalties)  for  the  four  possible  decision  outcomes  on  a  trial 
(correct  identification  of  signal  or  “hit”,  correct  identification  of  noise  or  “correct  rejection”, 
incorrect  signal  response  or  “false  alarm”,  incorrect  noise  response  or  “miss”),  one  can  define  an 
optimal  criterion  (or  threshold)  for  the  value  of  \nl(x).  That  optimal  rule  is 

respond  “signal”  iff  In  l(x )  >  In [/?]  [  1  ] 


where  (5  is  defined  by  (Tanner  &  Swets,  1 954) 


V  +V 

n  correct -rejection  false- alarm 

p  ~  V  +v 

hit  miss 


p{noise ) 
p(signal) 


[2] 


where  the  Vv  are  the  utilities  of  the  four  possible  outcomes,  and  p(noise)/p(signal)  is  the  prior 
odds  ratio  of  noise  to  signal. 
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The  Advice  Task 


Suppose  that  additional  information  were  available  from  an  uncorrelated,  external 
advisor,  A,  a  source  with  detection  sensitivity  d'A  and  criterion  f3A.  (For  discussion  of  correlated 
sources  and  information  aggregation,  see  Durlach,  Braida,  &  Ito,  1986;  Sorkin  &  Dai,  1994). 

How  and  when  should  information  from  advisor  A  be  incorporated  into  the  DM’s  decision?  In  the 
continuous  advice  case,  the  advisor  provides  an  estimate  of  signal  occurrence  that  is  summarized 
by  its  likelihood  estimate,  lA(x).  All  of  the  information  present  is  captured  by  combining  DM’s 
likelihood  estimate  with  the  advisor’s  likelihood  estimate,  in  In lf(xDM,  x^J.  If  the  DM  and  A 
sources  are  independent,  the  joint  likelihood  estimate  is  given  by  the  product  of  the  likelihood 
estimates  and 

In  U(xDM  ,xA)]  =  in  [lDM  (x)  •  lA  (x)]  =  ln[/DA,  (x)]  +  ln[/^  (x)]  [3] 

Then  DM’s  decision  rule  is  respond  “signal”  iff  ln[/^  (x)]  >  In  fi  —  ln[/DM  (x)]  [4] 

The  resulting  hit  probability  is  given  by  the  volume  under  the  bivariate  normal  signal  distribution, 
f{[\n[lDM(x),  In lA(x)]\s},  defined  by  the  area  above  the  diagonal  criterion  line  (a  line  with  slope 
equal  to  -1  and  crossing  the  abscissa  at  ln(/3)).  The  false  alarm  probability  is  given  by  the  (similar) 
volume  under  the  noise  distribution,  f{[\n[lDM(x),\T\lA(x)]\n}. 

If  accessing  the  information  has  a  cost,  DM  establishes  advice  purchase  criteria  on 
In Udm(x)].  That  is,  DM  only  purchases  the  advice  when  her  observation  is  above  a  value  In/?/  and 
below  a  value  ln/?2.  The  optimal  settings  for  these  criteria  can  be  determined  by  calculation  of  the 
expected  value  of  the  trial  outcome  minus  the  advice  cost. 

A  number  of  interesting  cases  result  when  the  information  from  A  is  binary  rather  than 
continuous  (advisor  A’s  response  is  either  “signal”  or  “noise”).  One  such  case  is  the  so-called 
“confirmation  bias”  problem:  DM  is  faced  by  the  option  of  choosing  one  of  two  advisors:  advisor 


fMUxm 


Figure  2.  Binary  advice  situation. 
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A  or  B.  Both  advisors  have  the  same  cost  and  expertise;  the  only  difference  is  the  response 
criteria  that  they  employ,  ln(7 3^)  and  In (/3b)-  Advisor  A  uses  a  liberal  (low)  criterion,  and  B  uses  a 
conservative  (high)  criterion.  The  question  is  how  DM’s  choice  of  advisor  should  depend  on  the 
magnitude  of  DM’s  initial  estimate.  It  can  be  shown  that  the  optimal  choice  after  making  a  high 
estimate  is  to  choose  the  liberal  advisor,  and  the  optimal  choice  after  having  made  a  low  estimate 
is  to  choose  the  conservative  advisor.  This  is  the  so-called  “confirmation”  strategy,  wherein  the 
advisor’s  recommendation  is  usually  expected  to  support  DM’s  initial  estimate. 

The  situation  can  be  understood  graphically  by  examining  figure  2.  As  in  the  continuous 
case,  DM’s  goal  is  to  maximize  the  volume  of  the  correct  regions  under  the  respective  signal  and 
noise  distributions.  How  should  the  DM’s  choice  of  an  advisor  depend  on  the  location  of  her 
likelihood  estimate  when  her  estimate  is  greater  than  \n(/3D/)  but  less  than  \n(j3D2)?  The  respective 
response  regions  for  the  two  advisors  are  indicated  by  the  horizontal  lines  in  Figure  2;  the  liberal 
advisor  responds  “signal”  when  his  estimate  is  above  ln(7?<),  and  the  conservative  advisor 
responds  “signal”  when  his  estimate  is  above  In (/3B).  The  following  consulting  strategy  is  optimal: 
If  the  DM’s  estimate  is  between  In (/3DI)  and  she  should  consult  the  conservative  advisor 
In  (fig),  if  her  estimate  is  between  \n(/3)  and  \n(/3Di),  she  should  consult  the  liberal  advisor  In  ((5a).  It 
can  be  seen  that  the  alternative  or  discontinuing  strategy,  of  reversing  the  choice  of  liberal  and 
conservative  advisors,  would  result  in  a  lower  probability  of  correct  responses.  (Note  the 
optimum  criterion  line  overlaid  on  the  diagram.  The  square  area  with  sides  \\u(Pa)  to  In ((3B)]  by 
[ln(7 3)  to  \n(pD2)]  would  be  un-shaded  in  the  disconfirming  strategy,  and  the  area  with  sides 
[In  (Pa)  to  In  (fiB)\  by  [In  ((3Di)  to  In  (Pd)]  would  now  be  a  shaded  region  resulting  in  an  increase  in 
false  alarm  rate  and  decrease  in  hit  rate.)  The  optimal  strategy  can  be  interpreted  as  a 
“confirmation”  strategy,  since  on  most  trials  the  DM  will  choose  the  advisor  who  is  most  likely  to 
agree  with  her  initial  estimate.  An  intuitive  explanation  of  this  is  to  realize  that  if  DM  has  made  a 
high  evaluation,  only  a  negative  (and  unlikely)  reply  from  the  liberal  advisor  should  cause  her  to 
reject  her  initial  estimate. 

Team  Decision  Tasks 


Our  approach  to  modeling  group  decision  making  shares  many  features  with  the  advice 
or  information  acquisition  problem.  We  assume  that  group  performance  depends  not  only  on  the 
members’  individual  expertise,  but  also  on  how  member  information  is  communicated, 
aggregated,  and  converted  to  the  group  decision.  Members  may  have  different  levels  of  expertise 
(effective  signal-to-noise  ratios),  biases  (response  criteria),  correlations  (degrees  of  shared  noise), 
and  response  rules.  We  assume  that  members  generally  update  their  estimates  in  an  optimal 
Bayesian  fashion.  However,  the  communication  of  member  information  is  assumed  to  be 
inherently  imperfect.  Figure  3  shows  a  general  diagram  of  such  a  system.  The  team  of  local 
decision  makers  (LDMs)  must  decide  about  the  occurrence  of  a  signal;  the  task  is  made  difficult 
by  noise  (both  common  and  shared)  and  by  the  LDMs’  having  a  discrete  response  vocabulary. 
After  receiving  the  input,  each  member  estimates  the  likelihood  of  signal  occurrence  and  converts 
that  estimate  to  a  categorical  response  ru.  That  response  is  communicated  to  the  Decision  Center 
(DC)  where  the  group's  final  decision  is  made.  The  DC  uses  a  set  algorithm,  such  as  a  specific 
majority  of  signal  or  noise  votes,  to  aggregate  the  members’  information.  The  DC  also  may 
employ  a  deliberation  process  that  involves  additional  interaction  and  feedback  among  the 
members. 
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<1  aorngwnL^1 


Figure  3.  f^(x,)  indicates  likelihood  ratio.] 


In  a  hypothetical  jury,  each  member  updates  her  opinion  (and  re-votes)  upon  hearing  the 
simultaneous,  non-anonymous  votes  of  the  other  jurors.  All  members  vote  at  once,  member 
judgments  are  updated  in  an  optimal  fashion,  and  the  updating  and  voting  process  is  repeated 
until  a  majority  or  stopping  criterion  is  reached.  The  updating  calculation  of  the  juror’s  response 
is  equivalent  to  the  juror’s  Bayesian  recalculation  of  likelihood  ratio.  It  is  assumed  that  each  juror 
is  knowledgeable  about  the  expertise  and  bias  of  the  other  jurors.  Under  some  conditions,  jury 
performance  can  closely  approximate  the  maximum  likelihood  aggregation  of  the  members’ 
continuous  responses  (Swaszek  &  Willett,  1995;  Sorkin  et  al.,  2004). 

An  interesting  and  important  deviation  from  the  jury  aggregation  model  occurs  when  the 
LDM  information  arrives  sequentially  rather  than  simultaneously;  such  as  in  a  group  of 
electronically  networked  LDMs.  We’ve  tested  the  effects  of  different  sequential  response  rules  on 
simulated  group  performance  and  have  shown  that  the  order  of  arrival  of  member  information  has 
a  potentially  large  effect  on  the  accuracy  and  speed  of  the  decision.  We  assume  a  member’s  vote 
is  conveyed  to  the  group  in  an  individual,  sequential  fashion.  As  each  member’s  response  is 
made,  the  other  members  update  their  individual  estimates  in  a  Bayesian  fashion.  A  fixed 
aggregation  rule  is  then  used  to  either  arrive  at  a  group  decision  via  a  hidden  vote.  Members  don’t 
speak  again  until  all  have  spoken.  This  ‘deliberation’  process  continues  until  a  group  decision  is 
made  or  time  runs  out.  Table  1  shows  some  possible  protocols  that  could  be  used  to  determine  the 
next  team  member  to  speak. 
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Tabic  1. 


The  next  speaker  to  access  the  network  is: 


Random 

Most/least  expert 
Most/1  east  biased 
Largest/smallest  estimate 
Most/least  confident 
Largest/smallest  weight 


Ran(i) 

Maxfd  Min(d‘J 
Max  \lnp^\,  Min  \lnf^\ 

Max  \lnJfxJl  Min  \lnAfxJ] 

Max  |  InAfxJ-lnfi  \,  Min  \  InAfxJ-lnfi 
Max  w/rj  where: 

P(ri,i\ so>] 

^/f'i)=ln[p(ru\sI)/(p(ru\s0)J 


Experimental  Method 

The  basic  decision  task  is  to  judge  whether  the  stimulus  present  on  an  experimental  trial 
was  generated  by  a  signal-plus-noise  or  noise-alone  condition.  Each  participant  is  presented  with 
a  graphical  display  like  the  set  of  analog  gauges  shown  in  figure  4,  and  the  group  must  report 
whether  the  displays  were  due  to  a  signal  or  noise.  The  setting  of  each  display  (vertical)  element 
is  determined  by  a  normal  distribution  whose  mean  depends  on  the  parameters  of  the  experiment. 
Versions  of  this  task  have  been  studied  extensively  in  our  laboratory  (Luan,  Sorkin,  &  Itzkowitz, 
2004;  Montgomery  and  Sorkin,  1996;  Sorkin  &  Dai,  1994;  Sorkin,  Hays,  &  West,  2001;  Sorkin, 
Mabry,  Weldon,  &  Elvers,  1991;  and  Sorkin,  West,  &  Robinson,  1998).  The  difficulty  of  the  task 
is  determined  by  the  display’s  physical  and  statistical  parameters.  The  major  physical  parameters 
are  the  display  duration  and  the  visual  angle  subtended  by  the  display.  The  statistical  factors  are  the 
means  of  the  signal-plus-noise  and  noise-alone  distributions  and  the  value  of  their  common 
standard  deviation.  In  our  experiments,  the  main  factor  that  determines  task  difficulty  is  the 
display  signal-to-noise  ratio  (SNR),  which  is  the  difference  between  the  distribution  means 
divided  by  their  (common)  standard  deviation.  The  individual  SNR  for  each  member  can  be 
controlled,  as  can  the  correlation  between  member  displays.  We  record  the  mean  value,  Xjj,  of  the 
trial-by-trial  display  presented  to  each  member  i  on  each  trial  j.  The  monetary  payoff  to  the 
participants  depends  on  the  accuracy  of  the  group ’s  detection  performance.  Sometimes  the 
monetary  payoff  also  depends  on  the  accuracy  of  the  individual  members’  performance  on  their 
first  response;  this  manipulation  serves  to  control  their  initial  response  criterion.  This  general 
decision  task  can  be  run  under  different  rules  for  member  interaction  and  information  display. 
More  complex  displays  can  be  employed  to  convey  the  expertise  and  criterion  of  information 
sources  and  the  correlation  of  different  sources. 


Figure  4.  Example  of  a  typical  display  (signal-plus-noise  trial). 
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Results 


Effect  of  Consistency  and  Correlation  of  Information  Sources 

This  experiment  tested  how  decision  makers  weigh  the  estimates  received  from  different 
sources  when  those  sources  vary  in  their  (a)  number,  (b)  reliability  or  expertise,  (c)  apparent 
consistency  (internal  correlation),  and  (d)  correlation  with  the  decision  maker’s  initial  estimate. 
The  human  decision  makers  performed  the  graphical  decision  task,  aided  by  simulated  team 
members.  The  decision  maker  observed  a  display  of  a  signal-plus-noise  or  noise-alone  event  and 
then  made  an  estimate  of  the  likelihood  of  signal  occurrence.  The  decision  maker  was  then 
supplied  with  a  graphical  display  of  several  estimates  made  by  one  or  two  sets  of  simulated  team 
members.  The  decision  maker  was  then  required  to  make  a  final  yes-no  decision  about  the 
occurrence  of  signal  on  that  trial.  The  payoff  to  the  decision  maker  was  based  on  the  accuracy  of 
the  final  yes-no  decision.  Performance  was  assessed  when  the  estimates  of  the  virtual  team 
members  were  either  pair-wise  uncorrelated  or  correlated  (among  themselves)  at  some  level,  or 
had  differing  overall  indices  of  detection.  By  computing  the  correlation  over  trials  between  the 
decision  and  the  source’s  magnitude,  the  experiment  assessed  how  much  decision  weight  the 
decision  maker  gave  to  each  information  source. 

One  experimental  condition  tested  which  of  two  equal-information  sources  (sources  with 
equal  aggregate-d’)  would  be  given  the  higher  weight:  the  one  with  the  internal  pair-wise 
correlation  and  higher  individual  d’s,  or  the  one  with  the  zero  pair-wise  correlation  and  lower 
component  d’s.  Thus,  this  condition  tested  for  the  presence  of  a  bias  toward  sub-source 
consistency  or  sub-source  expertise.  Further  conditions  were  run  comparing  sources  with 
differing  overall  informational  value.  The  results  of  these  experiments  indicated  that  there  is  a 
small  but  significant  bias  toward  information  sources  that  have  higher  consistency  and  higher 
component  expertise,  even  though  the  information  available  from  such  sources  is  identical  to  or 
less  than  that  received  from  lower  consistency  sources.  This  is  an  important  result  because  it 
shows  the  existence  of  a  here-to-fore  unreported  bias  toward  information  sources  that  have  higher 
component  expertise  or  internal  consistency.  Such  a  bias  will  result  in  inefficiency  in  decision 
performance;  performance  will  decrease  as  a  function  of  the  discrepancy  between  actual  and 
optimal  weighting  of  input  sources.  It  may  be  possible  to  provide  a  decision  maker  with  the 
estimates  received  from  different  sources  so  that  the  overall  information  display  adjusts  for  such  a 
pre-existing  weighting  bias. 

Effect  of  Correlation  Between  Decision-maker  and  External  Information  Sources 


When  information  from  two  external  sources  is  available  to  help  a  decision-maker  (DM) 
make  her  final  decision,  following  the  majority  choice  of  the  two  sources  and  decision-maker  can 
be  a  useful  decision  heuristic  for  rapidly  achieving  a  final  decision.  The  advantage  of  this 
heuristic  is  greatest  when  the  information  from  the  three  sources  arises  from  independent 
observations  and  the  sources  are  equal  in  expertise.  However,  when  one  of  the  source’s 
information  is  correlated  with  the  DM’s  (which  would  result  in  more  agreement  between  that 
source  and  the  DM),  the  statistical  advantage  of  information  aggregation  is  reduced.  Therefore,  as 
the  correlation  between  one  source  and  the  DM  increases,  the  DM  should  assign  less  final 
decisional  weight  to  that  source  (and  herself),  relative  to  the  estimate  from  the  other,  independent, 
source.  In  this  experiment,  two  information  sources  with  nearly  equal  detection  abilities  to  the 
participants  were  presented  after  the  DM  made  an  initial  estimate.  We  manipulated  the  correlation 
(pPA)  between  one  information  source  (A)  and  the  participants’  decision  estimates  at  three 
different  levels:  0,  0.4  and  0.7,  and  kept  the  other  source  (B)  independent  (ppb  =  0, 
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Pab  ~  0).  The  results  indicated  that  when  pPA  increased,  participants  (1)  assigned  significantly 
more  decision  weight  to  information  from  themselves  and  source  A,  and  failed  to  assign 
appropriate  decision  weight  to  source  B;  (2)  continued  to  use  the  simple  majority  decision  rule  to 
make  their  final  decision  and  usually  did  not  follow  source  B’s  estimates  when  B’s  estimates 
were  counter  to  the  estimates  of  the  participant  and  source  A;  and  (3)  consequently  had  reduced 
final  decision  efficiencies  as  pPA  was  increased.  We  concluded  that  the  simple  majority  decision 
heuristic  was  extensively  used  when  decision-makers  integrated  information  from  different 
sources  and  that  they  weighed  information  from  agreeing  (and  correlated)  sources  much  more 
than  information  from  disagreeing  sources.  This  type  of  bias  results  in  a  loss  in  the  benefit  of 
information  contained  in  the  disagreeing  source  as  well  as  in  a  reduced  accuracy  in  the  final 
decision. 

Information  Acquisition  . 

Seeking  advice  from  other  people  or  sources  is  a  common  practice  in  making  real-life 
decisions  and  in  command  and  control  decision  making.  We  developed  a  normative  model  that 
describes  how  advice-taking  should  depend  on  the  decision  maker’s  own  estimate,  as  well  as  the 
cost,  expertise,  and  decision  bias  of  the  potential  advisors.  Three  experiments  were  conducted  to 
examine  how  human  participants  took  and  utilized  advice  in  different  decision  environments,  and 
whether  their  observed  behaviors  were  consistent  with  the  model’s  prescriptions.  In  the 
experiments,  a  decision  maker  (DM)  first  observed  a  visual  stimulus  and  then  made  an  initial 
estimate  of  the  nature  of  the  stimulus:  signal-plus-noise  (S)  or  noise-alone  (N).  Then,  information 
from  external  sources  was  available  for  the  DM  to  purchase  regarding  the  decision  event.  After 
the  DM  made  her  consulting  decision  and  integrated  the  advice  with  her  own,  she  made  a  final 
decision.  On  each  trial  of  one  experiment,  participants  were  asked  to  make  a  decision  about 
whether  or  not  to  consult  an  advisor.  Both  the  cost  (High  or  Low)  and  the  displayed  content  of  the 
advice  were  manipulated  (Continuous  or  Binary).  It  was  found  that  participants  consulted  the 
advisor  more  frequently  when  their  own  estimates  were  uncertain  and  also  when  the  advice  was 
expressed  in  the  continuous  mode  rather  than  the  binary  mode. 

In  a  further  experiment,  two  advisors  with  equal  expertise  and  consulting  costs  but 
different  decision  biases  (criteria,  P)  were  presented.  Four  hypothesized  consulting  strategies 
were  first  tested  in  computer  simulations:  (a)  a  Low  Estimate  followed  by  a  choice  of  a  Liberal 
Advisor  or  a  High  Estimate  followed  by  a  choice  of  a  Conservative  Advisor,  (b)  a  Low  Estimate 
followed  by  a  choice  of  a  Conservative  Advisor  or  a  High  Estimate  followed  by  a  choice  of  a 
Liberal  Advisor,  (c)  Always  choose  Liberal  Advisor,  and  (d)  Always  choose  Conservative 
Advisor.  The  (a)  strategy  of  “Low-C-High  -L”,  which  has  a  similar  working  rationale  as  the  so- 
called  “confirmation-bias”  strategy,  was  the  best  in  simulations  and  theory  (see  figures  2. ,5.  and 
earlier  theoretical  description  of  the  Advice  problem).  This  strategy  prescribes  that:  if  a  DM  has 
an  observation  that  is  more  likely  to  lead  her  to  make  a  N  decision,  she  should  consult  a 
“Conservative”  advisor;  otherwise,  she  should  consult  a  “Liberal”  one.  Because  a  Conservative 
advisor  was  defined  in  this  study  as  the  advisor  who  was  biased  to  make  more  N  decisions  than  S 
(tended  to  give  more  N  than  S  opinions  to  the  DM)  and  a  Liberal  advisor  was  defined  to  do  the 
contrary,  “Low-C-High-L”  is  actually  a  “confirming”  strategy:  a  DM  should  always  consult  the 
advisor  who  is  more  likely  to  agree  with  her  own  decision  estimate.  The  results  are  shown  in 
figure  6.  Most  subjects  followed  the  optimal  strategy;  none  followed  the  non-optimal  strategy.  It 
can  be  concluded  that  people  have  certain  behavioral  tendencies  when  taking  advice  from 
external  sources  and  that  these  generally  confirm  to  optimal  behavior.  Our  advice  model  can  be 
used  to  identify  precisely  how  those  observed  behaviors  deviate  from  the  optimal,  and  may 
contribute  additional  insight  into  understanding  human  advice-acquisition  and  decision-making. 
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Figure  5.  The  expected  payoff  (EV)  as  the  function  of  k  (an  index  of  consulting  frequency)  resulted  from 
the  four  tested  strategies  in  the  optional  consulting  condition.  The  optimal  strategy  has  the  highest  payoff; 
always  choosing  one  source  is  in  between;  the  reverse  strategy  is  the  worst. 


Figure  6.  The  number  of  participants  who  used  each  consulting  strategy  in  the  two  experimental  conditions 
of  the  advisor  experiment.  (None  of  the  participants  used  the  Low-L-High-C  strategy). 
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Team  Deliberation 


A  series  of  experiments  studied  collaborative  detection  in  decision  making  groups. 
Collaboration  is  defined  as  the  process  by  which  group  members  sequentially  share  their  private 
information  with  the  group.  The  many  factors  that  influence  the  group  during  this  process  can  be 
grouped  into  two  categories:  group  composition  (the  accuracies,  biases,  and  correlation  of 
members’  opinions)  and  organizational  structure  (speaking  rules,  response  protocols,  and  decision 
criteria).  Using  a  normative  model,  we  studied  the  effects  of  the  team  decision  criteria  (simple 
majority  and  unanimity)  and  the  response  protocols  (member  properties  determining  who  speaks 
next),  as  well  as  the  effects  of  group  composition  and  organizational  structure  on  performance. 
Real  and  simulated  group  performance  was  evaluated  in  a  number  of  ways  (accuracy,  post- 
deliberative  bias,  deliberation  length,  and  deliberation  efficacy).  The  model  makes  some  very 
specific  assumptions:  members  attempt  to  optimize  the  group  payoff/performance,  members  are 
rational  decision  makers  with  appropriate  motivation,  and  members  update  their  estimates  and 
combine  information  from  other  group  members’  public  votes  in  a  Bayesian  manner.  The  general 
architecture  of  the  model  is  shown  in  figure  7. 
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The  response  sequence  protocol  is  employed  to  transmit  (to  all  members)  a  single 
vote  plus  information  about  the  voter  [f(d'h  fa  P:J  pLk )]  according  to  defined  rule 


Figure  7.  Computational  architecture  of  group  decision  process. 


One  experiment  tried  to  determine  which,  if  any,  response  protocols  were  used  in  groups 
when  members’  participated  according  to  a  voluntary  structure.  Because  we  desired  speaking 
orders  that  reflected  group  members’  desire  to  share  their  information  with  the  group,  we  set  the 
governing  response  protocol  to  sequence  speakers  in  ascending  order  of  their  response  time  to 
their  initial  stimulus.  Previous  research  indicated  that  group  members  who  are  certain  (or 
confident)  of  their  decision  are  the  most  likely  to  participate  (Carlston,  1977;  Sneizek  &  Henry, 
1989).  We  found  that  a  members’  observation  magnitude,  relative  to  their  own  individual 
decision  criterion,  had  a  small  but  clear  impact  on  the  speed  of  a  group  members’  decision. 


Team  deliberation,  even  with  a  requirement  for  unanimity,  was  relatively  rapid.  This  was 
consistent  with  the  predictions  of  Swaszek  &  Willett  (1995).  Second,  group  performance  was 
better  than  that  of  the  best  member  in  the  group,  providing  evidence  consistent  with  the  model 
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(and  simulations)  and  contrary  to  some  previous  human  studies  that  found  that  groups  rarely 
perform  better  than  their  best  member  (Gigone  &  Hastie  1997;  Einhom,  Hogarth,  &  Klempner, 
1977).  Third,  group  deliberation  with  a  requirement  for  unanimity  had  the  effect  of  “debiasing” 
the  most  extreme  group  members.  Even  when  two-thirds  of  the  group  members  began 
deliberation  with  an  initial  bias  (either  liberal  or  conservative),  the  group  decision  had  a  generally 
neutral  bias. 

All  of  the  groups  tested  had  higher  decision  performance  than  that  of  the  group’s  best 
member,  indicating  that  the  groups  were  using  the  information  provided  in  deliberation.  None  of 
the  protocols  produced  results  having  both  the  highest  detection  accuracy  and  the  minimum 
deliberation  duration.  Thus,  these  two  performance  metrics  are  not  optimized  by  the  same 
protocol.  Simulated  and  actual  groups  using  a  Minimum  d’  rule  and  a  unanimous  decision 
criterion  had  higher  detection  ability  than  the  other  groups.  Simulated  and  actual  groups  (see 
figure  8)  requiring  unanimity  and  using  a  Minimum  d’  protocol,  had  the  slowest  performance, 
e.g.,  highest  number  of  votes  than  other  groups. 


Figure  8.  Mean  deliberation  times  and  associated  standard  errors  for  actual  groups  (left  panel)  and 

simulated  groups  (right  panel).  The  left-most  column  is  for  a  simple  majority  rule;  the  right 
four  columns  are  for  unanimous  rale  groups  using,  respectively,  a  random,  minimum  d’, 
maximum  d’  and  maximum  observation-criterion  protocol. 

Conclusions  and  Possible  Applications 

It  is  clear  that  team  performance  is  a  function  both  of  the  team  members’  characteristics 
and  the  organizational  structure  used  to  reach  consensus  decisions.  Whereas  the  former  mostly 
affects  individuals’  private  judgments,  the  latter  affects  how  these  judgments  are  integrated  with 
information  received  from  other  team  members.  Our  approach  provides  an  integrated  model  for 
addressing  individual  and  team  decision  making  behavior  in  deliberative  groups  and  provides  a 
consistent  account  of  such  behavior  in  the  laboratory.  In  general,  individuals  within  the  groups 
tested  updated  their  own  opinions  according  to  the  normative,  Bayesian,  predictions.  Moreover, 
groups  closely  matched  the  predictions  of  group  accuracy  and  deliberation  length  provided  by 
simulated  groups.  Although  our  experimental  procedure  was  simplified  and  did  not  reflect  the 
complexity  of  actual  deliberative  teams,  our  results  indicate  that  groups  act  in  a  largely  rational 
way. 
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The  basic  issues  addressed  by  our  model  (and  the  additional  questions  raised  here)  have 
important  implications.  Because  so  many  important  decisions  are  made  by  both  face-to-face  and 
networked  deliberative  groups  (e.g.,  juries,  military  teams,  and  corporate  boards),  it  is  essential 
that  we  discover  ways  to  maximize  the  performance  of  these  groups.  In  some  situations,  it  is  clear 
that  adding  some  automated  control  of  the  “deliberation”  process  in  a  networked  group  could 
result  in  small  but  measurable  increases  in  accuracy  or  important  decreases  in  decision  time. 
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